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VEHICLE INTERIOR IDENTIFICATION AND MONITORING SYSTEM 



BACKGROUND OF THE INVENTION 

In 1984 the National Highway Traffic Safety 
Administration (NHTSA) of the U.S. Department of 
Transportation issued a requirement for frontal crash 
protection of automobile occupants. This regulation 
mandated "passive occupant restraints" for all passenger 
cars by 1992. A more recent regulation requires both 
driver and passenger side airbags on all passenger cars and 
light trucks by 1998. In addition, the demand for airbags 
is accelerating in both Europe and Japan and it is expected 
that, within a few years, all vehicles produced in these 
areas (36 million vehicles) and eventually worldwide (50 
million vehicles) will be equipped with airbags as standard 
equipment. 

Whereas thousands of lives h'ave been saved by airbags, 
significant improvements can be made. As discussed in 
detail in copending patent application 08/040,978 cross- 
referenced above, and included herein by reference, for a 
variety of reasons vehicle occupants can get too close to 
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the airbag before it deploys and can be seriously injured 
or killed. 

Also, a child in a rear facing child seat which is 
placed on the right front passenger seat is in danger of 
being seriously injured if the passenger airbag deploys. 
This has now become an industry wide concern and the US 
automobile industry is urgently searching for a solution 
which will prevent the deployment of the passenger side 
airbag if a rear facing child seat is present. An 
improvement on the invention disclosed in the referenced 
copending patent application as will be disclosed in 
greater detail below, includes more sophisticated means to 
identify objects within the passenger compartment and will 
solve this problem. 

Initially these systems will solve the out-of -position 
occupant and the rear facing child seat problems related to 
current airbag systems and prevent unneeded deployments 
when a seat is unoccupied. Airbags are now under 
development to protect rear seat occupants in vehicle 
crashes. A system will therefore be needed to detect the 
presence of occupants, determine if they are out-of- 
position and to identify the presence of a rear facing 
child seat in the rear seat. Future automobiles can be 
expected to have eight or more airbags as protection is 
sought for rear seat occupants and from side impacts. In 
addition to eliminating the disturbance of unnecessary 
airbag deployments, the cost of replacing these airbags 
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will be excessive if they all deploy in an accident. The 
improvements described below minimize this cost by not 
deploying an airbag for a seat which is not occupied by a 
human being • An occupying item of a seat may be a living 
occupant such as a human being or dog, another living 
organism such as a plant, or an inanimate object such as a 
box or bag of groceries. 

A device to monitor the vehicle interior and identify 
its contents is needed to solve these and many other 
problems. For example, once a Vehicle Interior 
identification and Monitoring System (VIMS) for identifying 
and monitoring the contents of a vehicle is in place, many 
other products become possible including the following: 

Inflators now exist which will adjust the amount of gas 
flowing to the airbag to account for the size and position 
of the occupant and for the severity of the accident. The 
vehicle identification and monitoring system of this 
invention will control such inflators based on the presence 
and position of vehicle occupants or of a rear facing child 
seat . 

Side impact airbag systems will begin appearing on 1995 
vehicles. The danger of deployment induced injuries will 
exist for side impact airbags as they now do for frontal 
impact airbags. A child with his head against the airbag 
is such an example. The system of this invention will 
minimize such injuries. 



Future vehicles may be provided with a standard 
cellular phone as well as the Global Positioning System 
(GPS) , an automobile navigation or location system which is 
scheduled to be available on at least one vehicle model in 
1998. In the event of an accident, the phone may 
automatically call 911 for emergency assistance and report 
the exact position of the vehicle. If the vehicle also has 
a system as described below for monitoring each seat 
location, the number and perhaps the condition of the 
occupants could also be reported. In that way the 
emergency service (EMS) would know what equipment and how 
many ambulances to send to the accident site. 

Vehicle entertainment system engineers have stated that 
the quality of the sound in the vehicle could be improved 
if the number, size and location of occupants and other 
objects were koown. This information can be provided by 
the vehicle interior identification and monitoring system 
of this invention. 

Similarly to the entertainment system, the heating, 
ventilation and air conditioning system (HVAC) could be 
improved if the number, attributes and location of vehicle 
occupants were known. This can be used to provide a 
climate control system tailored to each occupant, for 
example, or the system can be turned off for certain seat 
locations if there were no occupant present. 

In some cases the position of a particular part of the 
occupant is of interest such as: (a) his hand or arm and 
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whether it is in the path of a closing window so that the 
motion of the window needs to be stopped; (b) the position 
of the shoulder so that the seat belt anchorage point can 
be adjusted for the best protection of the occupant; or, 
(c) the position of the rear of the occupants head so that 
the headrest can be adjusted to minimize whiplash injuries 
in rear impacts. 

The above applications illustrate the wide range of 
opportunities which become available if the identity and 
location of various objects and occupants, and some of 
their parts, within the vehicle were known. Once the 
system is operational it would be logical for the system to 
also incorporate the airbag electronic sensor and 
diagnostics system (SDM) since it needs to interface with 
SDM anyway and since they could share computer capabilities 
which will result in a significant cost saving to the auto 
manufacturer. For the same reasons, it would be logical 
for VIMS to include the side impact sensor and diagnostic 
system. As the VIMS improves to where such things as the 
exact location of the occupants ears and eyes can be 
determined, even more significant improvements to the 
entertainment system become possible through the use of 
noise canceling sound, and the rear view mirror can be 
automatically adjusted for the driver's eye location. 
Another example involves the monitoring of the driver's 
behavior over time which can be used to warn a driver if he 
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or she is falling asleep, or to stop the vehicle if the 
driver loses the capacity to control it. 

Using an advanced VIMS, as explained below, the 
position of the driver's eyes can be accurately determined 
and portions of the windshield can be selectively darkened 
to eliminate the glare from the sun or oncoming vehicle 
headlights- This system uses electro-chromic glass, a 
liquid crystal coating on the glass, or other appropriate 
technology, and detectors to detect the direction of the 
offending light source. In addition to eliminating the 
glare, the sun visor can now also be eliminated. 

The present invention adds more sophisticated pattern 
recognition capabilities such as fuzzy logic systems, 
neural network systems or other pattern recognition 
computer based algorithms to the occupant position 
measurement system disclosed in the above reference 
copending patent application and greatly extends the areas 
of application of this technology. An example of such a 
pattern recognition system using neural networks using 
sonar is discussed in two papers by Gorman, R, . P. and 
Sejnowski, T. J. Analysis of Hidden Units in a Layered 
Network Trained to Classify Sonar Targets 11 , Neural Networks , 
Vol.1, pp 75-89, 1988, and ' ' Learned Classification of 
Sonar Targets Using a Massively Parallel Network 11 , IEEE 
Transactions on Acoustics, Speech, and Signal Processing, 
Vol. 36, No. 7, July 1988. 
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"Pattern recognition" as used herein will mean any 
system which processes a signal that is generated by an 
object, or is modified by interacting with an object, in 
order to determine which one of a set of classes that the 
object belongs to. Such a system might determine only that 
the object is or is not a member of one specified class, or 
it might attempt to assign the object to one of a larger 
set of specified classes, or find that it is not a member 
of any of the classes in the set. The signals processed 
are generally electrical signals coming from transducers 
which are sensitive to either acoustic or electromagnetic 
radiation and if electromagnetic they can be either visible 
light, infrared, ultraviolet or radar. 

"To identify" as used herein will mean to determine 
that the object belongs to a particular set or class. The 
class may be one containing all rear facing child seats, 
one containing all human occupants, or all human occupants 
not sitting in a rear facing child seat depending of the 
purpose of the system. In the case where a particular 
person is to be recognized, the set or class will contain 
only a single element, the person to be recognized. 
Some examples follow: 

In a passive infrared system a detector receives 
infrared radiation from an object in its field of view , in 
this case the vehicle occupant, and determines the 
temperature of the occupant based on the infrared 
radiation. The VIMS can then respond to the temperature of 
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the occupant, which can either be a child in a rear facing 
child seat or a normally seated occupant, to control some 
other system. This technology could provide input data to 
a pattern recognition system but it has limitations related 
to temperature. The sensing of the child could pose a 
problem if the child is covered with blankets. It also 
might not be possible to differentiate between a rear 
facing and forward facing child seat. In all cases, the 
technology will fail to detect the occupant if the ambient 
temperature reaches body temperature as it does in hot 
climates. Nevertheless, for use in the control of the 
vehicle climate, for example, a passive infrared system 
which permits an accurate measurement of each occupant's 
temperature is useful. 

In a laser optical system an infrared laser beam is 
used to momentarily illuminate an object, occupant or child 
seat in the manner as described, and illustrated in FIG 8, 
of the co-pending patent application cross referenced 
above. In some cases a charge coupled device (a type of TV 
camera) is used to receive the reflected light. The laser 
can either be used in a scanning mode, or, through the use 
of a lens, a cone of light can be created which covers a 
large portion of the object. In each case a pattern 
recognition system, as defined above, is used to identify 
and classify, and can be used to locate, the illuminated 
object and its constituent parts. This system provides the 
most information about the object and at a rapid data rate. 
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Its main drawback is cost: which is considerably above that 
of ultrasonic or passive infrared systems. As the cost of 
lasers comes down in the future, this system will become 
more competitive. Depending on the implementation of the 
system, there may be some concern for the safety of the 
occupant if the laser light can enter the occupant's eyes. 

Radar systems have similar properties to the laser 
system discussed above. The wave length of a particular 
radar system can limit the ability of the pattern 
recognition system to detect object features smaller than a 
certain size. Once again, however , there is some concern 
about the health effects of radar on children and other 
occupants. This concern is expressed in various reports 
available from the United States Food and Drug 
Administration Division of Devices. 

The ultrasonic system is the least expensive and 
potentially provides less information than the laser or 
radar systems due to the delays resulting from the speed of 
sound and due to the wave length which is considerably 
longer than the laser systems. The wave length limits the 
detail which can be seen by the system. In spite of these 
limitations, as shown below, ultrasonics can provide 
sufficient timely information to permit the position and 
velocity of an occupant to be accurately known and, when 
used with an appropriate pattern recognition system, it is 
capable of positively determining the presence of a rear 
facing child seat. One pattern recognition system which 



9 



has been used to identify a rear facing child seat uses 
neural networks and is similar to that described in the 
above referenced papers by Gorman et al. 

A focusing system, such as used on some camera systems, 
could be used to determine the initial position of an 
occupant but is too slow to monitor his position during a 
crash. This is a result of the mechanical motions required 
to operate the lens focusing system. By itself it cannot 
determine the presence of a rear facing child seat or of an 
occupant but when used with a charge coupled device plus 
some infrared illumination for night vision, and an 
appropriate pattern recognition system, this becomes 
possible. 

From the above discussion, it can be seen that the 
addition of sophisticated pattern recognition means to any 
of the standard illumination and/or reception technologies 
for use in a motor vehicle permits the development of a 
host of new products, systems or capabilities heretofore 
not available and as described in more detail below. 

SUMMARY OF THE INVENTION 

This invention is a system to identify and monitor 
occupants and their parts and other objects in the 
passenger compartment of a motor vehicle, such as an 
automobile or truck, by processing the signal received from 
said occupants and their parts and other objects using one 
or more of a variety of pattern recognition techniques and 
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illumination technologies. The received signal may be a 
reflection of a transmitted signal, the reflection of some 
natural signal within the vehicle, or may be some signal 
emitted naturally by the object. Information obtained by 
the identification and monitoring system is then used to 
affect the operation of some other system in the vehicle. 
The principle objects and advantages are: 

1. To recognize the presence of a human on a 
particular seat of a motor vehicle and to use this 
information to affect the operation of another vehicle 
system such as the airbag system, heating and air 
conditioning system, or entertainment system, among others. 

2. To recognize the presence of a human on a 
particular seat of a motor vehicle and then to determine 
his/her position and to use this position information to 
affect the operation of another vehicle system. 

3. To recognize the presence of a human on a 
particular seat of a motor vehicle and then to determine 
his/her velocity relative to the passenger compartment and 
to use this velocity information to affect the operation of 
another vehicle system. 

4. To determine the position of a seat in the vehicle 
using sensors remote from the seat and to use that 
information in conjunction with a memory system and 
appropriate actuators to position the seat to a 
predetermined location. 
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5. To determine the position, velocity or size of an 
occupant in a motor vehicle and to utilize this information 
to control the rate of gas generation, or the amount of gas 
generated, by an airbag inflator system. 

6. To determine the fact that an occupant is not 
restrained by a seatbelt and therefore to modify the 
characteristics of the airbag system- This determination 
can be done either by monitoring the position of the 
occupant or through the use of a resonating device placed 
on the shoulder belt portion of the seatbelt* 

7. To determine the presence or position of rear 
seated occupants in the vehicle and to use this information 
to affect the operation of a rear seat protection airbag 
for frontal impacts. 

8. To determine the presence or position of occupants 
relative to the side impact airbag systems and to use this 
information to affect the operation of a side impact 
protection airbag system* 

9. To determine the openness of a vehicle window and 
to use that information to affect another vehicle system. 

10. To determine the presence of an occupant 1 s hand 
or other object in the path of a closing window and to 
affect the window closing system. 

11. To remotely determine the fact that a vehicle 
door is not tightly closed using an illumination 
transmitting and receiving system such as one employing 
electromagnetic or acoustic waves. 
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12. To determine the position of the shoulder of a 
vehicle occupant and to use that information to control the 

^seatbelt anchorage point. 

13. To determine the position of the rear of an 
occupant's head and to use that information to control the 
position of the headrest. 

14. To recognize the presence of a rear facing 
child seat on a particular seat of a motor vehicle and to 
use this information to affect the operation of another 
vehicle system such as the airbag system. 

15. To determine the total number of occupants of a 
vehicle and in the event of an accident to transmit that 
information, as well as other information such as the 
condition of the occupants, to a receiver remote from the 
vehicle . 

16. To affect the vehicle entertainment system based 
on a determination of the size or location of various 
occupants or other objects within the vehicle passenger 
compartment 

17. To affect the vehicle heating, ventilation and 
air conditioning system based on a determination of the 
number, size and location of various occupants or other 
objects within the vehicle passenger compartment. 

18. To determine the temperature of an occupant based 
on infrared radiation coming from that occupant and to use 
that information to control the heating, ventilation and 
air conditioning system. 
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19. To provide a vehicle interior monitoring system 
for determining the location of occupants within the 
vehicle and to include within the same system various 
electronics for controlling an airbag system. 

20. To determine the approximate location of the eyes 
of a driver and to use that information to control the 
position of the rear view mirrors of the vehicle. 

21. To monitor the position of the head of the 
vehicle driver and determine whether the driver is falling 
asleep or otherwise impaired and likely to lose control of 
the vehicle and to use that information to affect another 
vehicle system. 

22. To monitor the position of the eye lids of the 
vehicle driver and determine whether the driver is falling 
asleep or otherwise impaired and likely to lose control of 
the vehicle and to use that information to affect another 
vehicle system. 

23. To determine the location of the eyes of a 
vehicle occupant and the direction of a light source such 
as the headlights of an oncoming vehicle or the sun and to 
cause a filter to be placed in such a manner as to reduce 
the intensity of the light striking the eyes of the 
occupant. 

24. To determine the location of the eyes of a 
vehicle occupant and the direction of a light source such 
as the headlights of a rear approaching vehicle or the sun 
and to cause a filter to be placed in such a manner as to 
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reduce the intensity of the light reflected from the rear 
view mirrors and striking the eyes of the occupant. 

25. To determine the location of the ears of a 
vehicle occupant and to use that information to control the 
entertainment system so as to improve the quality of the 
sound reaching the occupant's ears through such methods as 
noise canceling sound* 

26. To recognize a particular driver based on such 
factors as physical appearance or other attributes and to 
use this information to control another vehicle system such 
as a security system, seat adjustment, or maximum permitted 
vehicle velocity, among others. 

These and other objects and advantages will become 
apparent from the following description of the preferred 
embodiments of the vehicle interior identification and 
monitoring system of this invention 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a side planer view with parts cutaway and 
removed of a vehicle showing the passenger compartment 
containing a rear facing child seat on the front passenger 
seat and a preferred mounting location for an occupant and 
rear facing child seat presence detector. 

FIG. 2 is a side planer view with parts cutaway and 
removed showing schematically the interface between the 
vehicle interior monitoring system of this invention and 
the vehicle cellular communication system. 
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FIG . 3 is a side planer view with parts cutaway and 
removed showing schematically the interface between the 
vehicle interior monitoring system of this invention and 
the vehicle heating and air conditioning system. 

FIG. 4 . is a side planer view with parts cutaway 
and removed showing schematically the interface between 
the vehicle interior monitoring system of this invention 
and the vehicle airbag system. 

FIG. 5 is a side planer view with parts cutaway and 
removed showing schematically the interface between the 
vehicle interior monitoring system of this invention and 
the vehicle entertainment system. 

FIG. 6 is a side planer view with parts cutaway and 
removed of a vehicle showing the passenger compartment 
containing a driver and a preferred mounting location for 
an occupant identification system. 

FIG. 7A is a functional block diagram of the ultrasonic 
imaging system illustrated in FIG. 1 using a 
microprocessor . 

FIG. 7B is a functional block diagram of the ultrasonic 
imaging system illustrated in FIG. 1 using an application 
specific integrated circuit (ASIC) . 

FIG. 8 is a side planer view with parts cutaway and 
removed of a vehicle showing the passenger compartment 
containing a box on the front passenger seat and a 
preferred mounting location for an occupant and rear facing 
child seat presence detector. 
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FIG. 9 is a side planer view with parts cutaway and 
removed of a vehicle showing the passenger compartment 
containing a driver and a preferred mounting location for 
an occupant position sensor for use in side impacts and 
also of a rear of occupant 1 s head locator for use with a 
headrest adjustment system to reduce whiplash injuries in 
rear impact crashes. 

FIG. 10 is a side planer view with parts cutaway and 
removed of a vehicle showing the passenger compartment 
containing a front passenger and a preferred mounting 
location for an occupant head detector and a preferred 
mounting location of an adjustable microphone and speakers* 

FIG. 11 is a side planer view with parts cutaway and 
removed of a subject vehicle and an oncoming vehicle, 
showing the headlights of the oncoming vehicle and the 
passenger compartment of the subject vehicle, containing 
detectors of the drivers eyes and detectors for the 
headlights of the oncoming vehicle and the selective 
filtering of the light of the approaching vehicle's 
headlights through the use of electro-chromic glass in the 
windshield. 

FIG. 12 is a side planer view with parts cutaway and 
removed of a vehicle and a following vehicle showing the 
headlights of the following vehicle and the passenger 
compartment of the leading vehicle containing a driver and 
a preferred mounting location for driver eyes and following 
vehicle headlight detectors and the selective filtering of 
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the light of the following vehicle's headlights through the 
use of electro-chromic glass in the rear view mirror. 

FIG. 13 is a side planer view with parts cutaway and 
removed of a vehicle showing the passenger compartment 
containing a driver, a shoulder height sensor and a 
seatbelt anchorage adjustment system. 

FIG . 14 is a side planer view with parts cutaway and 
removed of a seat in the passenger compartment of a 
vehicle showing the use of ultrasonic resonators to 
determine the position of the seat. 

FIG. 15 is a side planer view with parts cutaway and 
removed of the passenger compartment of a vehicle showing 
the use of ultrasonic resonators to determine the position 
of the driver seatbelt. 

FIG. 16 is a side planer view with parts cutaway and 
removed of the passenger compartment of a vehicle showing 
the use of an ultrasonic resonator to determine the 
openness of the driver window and of a system for 
determining the presence of an object, such as the hand of 
an occupant, in the window opening. 

FIG* 17 is a side planer view with parts cutaway and 
removed of the passenger compartment of a vehicle showing 
the use of an ultrasonic resonator to determine the 
openness of the driver side door. 

FIG. 18 is a side planer view with parts cutaway and 
remov d showing schematically the interface between the 
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vehicle interior monitoring system of this invention and 
the vehicle security system. 

FIG. 19 is a side planer view with parts cutaway and 
removed showing schematically the interface between the 
vehicle interior monitoring system of this invention and an 
instrument panel mounted inattentiveness warning light or 
buzzer and reset button- 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

In FIG. 1 a side planer view, with parts cutaway and 
removed, of a vehicle showing the passenger compartment 
containing a rear facing child seat 110 on the front 
passenger seat 120 and a preferred mounting location for an 
occupant and rear facing child seat presence detector is 
illustrated. In this implementation three ultrasonic 
transducers 131, 132 and 133 are used. Transducer 132 
transmits ultrasonic energy toward the front passenger seat 
which is reflected, in this case by a rear facing child 
seat 110, and the reflected waves are received by the 
transducers 131 and 133. The signal received by 
transducers 131 and 13 3 varies with time depending on the 
shape of the object occupying the passenger seat, in this 
case a rear facing child seat 110. Each object will 
reflect back a signal having a different pattern. Also, 
the pattern received by transducer 131 will differ slightly 
from the pattern received by transducer 133. Through the 
use of two transducers a sort of stereographic image is 
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received by the two transducers and recorded for analysis 
by processor 101 . This image will differ for each object 
which is placed on the vehicle seat and it will also change 
for each position of a particular object and for each 
position of the vehicle seat. 

The ^ image 1 ' recorded from each ultrasonic receiver is 
actually a time series of digitized data of the amplitude 
of the received signal versus time. Since there are two 
receivers, two time series are obtained which are processed 
by the processor 101. When different objects are placed on 
the front passenger seat the two images are different but 
there are also similarities between all images of rear 
facing child seats, for example, regardless of where on the 
vehicle seat it is placed and regardless of what company 
manufactured the child seat. Alternately, there will be 
similarities between all images of people sitting on the 
seat regardless of what they are wearing, their age or 
size. The problem is to find the % * rules. 11 which 
differentiate the occupant images from the rear facing 
child seat images. The similarities of these images for 
various child seats are frequently not obvious to a person 
looking at plots of the time series and thus computer 
algorithms are developed to sort out the various patterns. 

The determination of these rules is central to the 
pattern recognition techniques used in this invention. In 
general three approaches have been useful, artificial 
intelligence, fuzzy logic and artificial neural networks. 
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In some implementations of this invention, such as the 
determination that there is an object in the path of a 
closing window as described below, the rules are 
sufficiently obvious that a trained researcher can look at 
the returned acoustic signals and devise a simple algorithm 
to make the required determinations. In others, such as 
the determination of the presence of a rear facing child 
seat or of an occupant, artificial neural networks are used 
to determine the rules. One such set of neural network 
software for determining the pattern recognition rules is 
available from the NeuralWare Corporation of Pittsburgh, 
Pennsylvania . 

The system used for the determination of the presence 
of a rear facing child seat, of an occupant or of an empty 
seat was the artificial neural network. In this case the 
network operates on the- two- returned signals as sensed by 
transducers 131 and 133. Through a training session the 
system is taught to differentiate between the three cases. 
This is done by conducting a large number of experiments 
where all possible child seats are placed in all possible 
orientations on the front passenger seat. Similarly a 
sufficiently large number of experiments are run with human 
occupants and with boxes, bags of groceries and other 
objects. As many as 1000 such experiments are run before 
the neural network is sufficiently trained so that it can 
differentiate among the three cases and output the correct 
decision with a very high probability. 
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Once the network is determined, it is possible to 
examine the result using tools supplied by NeuralWare, for 
example, to determine the rules which were finally arrived 
at by the trial and error techniques. In that case, the 
rules can then be programmed into a microprocessor 
resulting in a fuzzy logic or other rule based system. 
Alternately, a neural computer can be used to implement the 
net directly. In either case, the implementation can be 
carried out by those skilled in the art of pattern 
recognition* If a microprocessor is used, a memory device 
is also required to store the data from the analog to 
digital converters which digitize the data from the 
receiving transducers. On the other hand, if a neural 
network computer is used, the analog signal can be fed 
directly from the transducers to the neural network input 
nodes and an intermediate memory is not required. Memory 
of some type is needed to store the computer programs in 
the case of the microprocessor system and if the neural 
computer is used for more than one task, a memory is needed 
to store the network specific values associated with each 
task. 

An alternate system is shown in FIG. 2 which is a side 
planer view showing schematically the interface between the 
vehicle interior monitoring system of this invention and 
the vehicle cellular communication system. In this view, 
an occupant 210 is shown sitting on the front passenger 
seat 220 and two ultrasonic transducers 231 and 232 are 
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used. One of the transducers 232 in this case acts as both 
a transmitter and receiver while transducer 231 acts only 
as a receiver. Alternately, transducer -231 could serve as 
both a transmitter and receiver or the transmitting 
function could be alternated between the two devices. As 
was also the case in FIG. 1 the transducers 231 and 232 are 
attached to the vehicle buried in the A-pillar trim, where 
their presence is disguised, and are connected to processor 
101 which is also hidden in the trim. The A-pillar is the 
roof support pillar which is closest to the front of the 
vehicle and which, in addition to supporting the roof, also 
supports the front windshield and the front door. 

The interface between the monitoring system and the 
cellular phone system is shown schematically by box 240 
which outputs to an antenna 250. The transducers 231 and 
232 in conjunction with the pattern recognition hardware 
and software, which is implemented in processor 101 and is 
packaged on a printed circuit board or flex circuit along 
with the transducers 231 and 232, determine the presence of 
an occupant within a few seconds after the vehicle is 
started. Similar systems located to monitor the remaining 
seats in the vehicle, also determine the presence of 
occupants at the other seating locations and this result is 
stored in the computer memory which is part of each 
monitoring system processor 101. In the event of an 
accident, the electronic system associated with the 
cellular phone system interrogates the various interior 
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monitoring system memories and arrives at a count of the 
number of occupants in the vehicle, and in more 
sophisticated systems, even makes a determination as to 
whether each occupant was wearing a seatbelt and if he or 
she is moving after the accident. The phone system then 
automatically dials the EMS operator and the information 
obtained from the interior monitoring systems is forwarded 
so that a determination can be made as to the number of 
ambulances and other equipment to send to the accident 
site. Vehicles having this capability are not yet in 
service but are planned for the late 1990 f s. Such vehicles 
will also have a system, such as the global positioning 
system, which permits the vehicle to determine its exact 
location and to forward this information to the EMS 
operator. 

The control of the heating, ventilating, and air 
conditioning (HVAC) system alone would probably not justify 
the implementation of an interior monitoring system at 
least until the time comes when electronic heating and 
cooling systems replace the conventional systems now used. 
Nevertheless, if the monitoring system is present it can be 
used to control the HVAC for a small increment in cost. 
The advantage of such a system is that since most vehicles 
contain only a single occupant, there is no need to direct 
heat or air conditioning to unoccupied seats. This permits 
the most rapid heating or cooling for the driver when the 
vehicle is first started and he is alone without heating or 
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cooling unoccupied seats. Since the HVAC system does 
consume energy, an energy saving also results by only 
heating and cooling the driver when he is alone. 

FIG. 3 shows a side planer view of a vehicle passenger 
compartment showing schematically the interface 260 between 
the vehicle interior monitoring system of this invention 
and the vehicle heating and air conditioning system. Like 
numbers in FIG. 3, and the other figures herein, have the 
same meaning. In addition to the acoustic transducers 231 
and 232, an infrared sensor 234 is also shown which 
monitors the temperature of the occupant. The output from 
each of the transducers is fed into processor 101 which is 
in turn connected to interface 260. In this manner the 
HVAC control is based on the occupant's temperature rather 
than that of the ambient air in the vehicle. This also 
permits each vehicle occupant to be independently monitored 
and the HVAC system to be adjusted for each occupant either 
based on a set temperature for all occupants or, 
alternately, each occupant could be permitted to set his 
own preferred temperature through adjusting a control knob 
shown schematically as 250 in FIG. 3. Since the monitoring 
system is already installed in the vehicle with its own 
associated electronics including processor 101, the 
infrared sensor can be added with little additional cost 
and can share the processing unit. 

The use of the vehicle interior monitoring system to 
control the deployment of an airbag is discussed in detail 
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in the co-pending patent application cross referenced 
above. In that case, the control is based on the use of a 
simple pattern recognition system to differentiate between 
the occupant and his extremities in order to provide an 
accurate determination of the position of the occupant 
relative to the airbag. If the occupant is sufficiently 
close to the airbag module that he is more likely to be 
injured by the deployment itself than by the accident, the 
deployment of the airbag is suppressed. This process is 
carried further by the interior monitoring system described 
herein in that the nature of the object occupying the 
vehicle seat is used to contribute to the airbag deployment 
decision. FIG. 4 shows a side planer view illustrating 
schematically the interface between the vehicle interior 
monitoring system of this invention and the vehicle airbag 
system 27 0. 

In this case an ultrasonic transducer 232 transmits a 
burst of ultrasonic waves which travel to the occupant 
where they are reflected back to receptors 231 and 232. 
The time period required for the waves to travel from the 
generator and return is used to determine the distance of 
the occupant to the airbag as described in the 
aforementioned co-pending patent application. In the case 
of this invention, however, the portion of the return 
signal which represents the occupants head or chest has 
been determined based on pattern recognition techniques 
such as a neural network. The relative velocity of the 
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occupant toward the airbag can then be determined, from 
successive position measurements, which permits a 
sufficiently accurate prediction of the time when the 
occupant would become proximate to the airbag. By 
comparing the occupant relative velocity to the integral of 
the crash deceleration pulse, a determination as to whether 
the occupant is being restrained by a seatbelt can also be 
made which then can affect the airbag deployment initiation 
decision. Alternately, the mere knowledge that the 
occupant has moved a distance which would not be possible 
if he were wearing a seatbelt gives information that he is 
not wearing one. 

A more detailed discussion of this process and of the 
advantages of the various technologies, such as acoustic or 
electromagnetic, can be found in SAE paper 940527, 
''Vehicle Occupant Position Sensing 11 by Breed et al, which 
is included herein by reference. In this paper it is 
demonstrated that the time delay required for acoustic 
waves to travel to the occupant and return does not prevent 
the use of acoustics for position measurement of occupants 
during the crash event. For position measurement and for 
many pattern recognition applications, ultrasonics is the 
preferred technology due to the lack of adverse health 
effects and the low cost of ultrasonic systems compared 
with either laser or radar. The main limiting feature of 
ultrasonics is the wave length which places a limitation on 
the size of feature which can be discerned. Optical 
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systems, for example, are required when the identification 
of particular individuals is required. 

It is well known among acoustics engineers that the 
quality of sound coming from an entertainment system can be 
substantially affected by the characteristics and contents 
of the space in which it operates and the surfaces 
surrounding that space. When an engineer is designing a 
system for an automobile he has a great deal of knowledge 
about that space and of the vehicle surfaces surrounding 
it. He has little knowledge of how many occupants are 
likely to be in the vehicle on a particular day, however, 
and therefore the system is a compromise. If the system 
knew the number and position of the vehicle occupants, and 
maybe even their size, then adjustments could be made in 
the system output and the sound quality improved. FIG. 5, 
therefore, illustrates schematically the interface between 
the vehicle interior monitoring system of this invention 
and the vehicle entertainment system 28 0. The particular 
design of the entertainment system which uses the 
information provided by the monitoring system can be 
determined by those skilled in the appropriate art. 

The maximum acoustic frequency which is practical to 
use for acoustic imaging in the systems is about 4 0 to 60 
kilohertz (KHz) . The wave length of a 50 KHz acoustic wave 
is about .6 cm which is too coarse to determine the fine 
features of a person's face, for example. It is well 
understood by those skilled in the art that features which 
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are smaller that the wavelength of the illuminating 
radiation cannot be distinguished. Similarly the wave 
length of common radar systems varies from about .9 cm (for 
33,000 MHz K band) to 133 cm (for 225 MHZ P band) which is 
also too coarse for person identification systems. In FIG, 
6, therefore, the ultrasonic transducers of the previous 
designs are replaced by laser transducers 231 and 232 which 
are connected to a microprocessor 101. In all other 
manners the system operates the same. The design of the 
electronic circuits for this laser system is described in 
some detail in the co-pending patent application cross- 
referenced above and in particular FIG. 8 thereof and the 
corresponding description. In this case a pattern 
recognition system such as a neural network system is 
employed and uses the demodulated signals from the 
receptors 231 and 2 32. — 

The output of processor 101 of the monitoring system is 
shown connected schematically to a general interface 290 
which can be the vehicle ignition enabling system; the 
entertainment system; the seat, mirror, suspension or other 
adjustment systems; or any other appropriate vehicle 
system. 

There are two preferred methods of^ implementing the 
vehicle interior monitoring system of this invention, a 
microprocessor system and an application specific 
integrated circuit system (ASIC) . Both of these systems 
are represented schematically as either 101 or 601 herein. 
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A block diagram illustrating the microprocessor system is 
shown in FIG. 7A which shows the implementation of the 
system of FIG. 1. An alternate implementation of the FIG. 
1 system using an ASIC is shown in FIG. 7B. In both cases 
the target, which may be a rear facing child seat, is shown 
schematically as 110 and the three transducers as 131, 132, 
and 133. 

In FIG. 8 a view of the system of FIG. 1 is illustrated 
with a box 295 shown on the front passenger seat in place 
of the rear facing child seat. The vehicle interior 
monitoring system of this invention is trained to recognize 
that this box 295 is neither a rear facing child seat nor 
an occupant and therefore it is treated as an empty seat 
and the deployment of the airbag is suppressed. This 
training is accomplished using a neural network with the 
commercially available software disclosed above and 
provided by NeuralWare of Pittsburgh. The system assesses 
the probability that the box is a person, however, and if 
there is even the remotest chance that it is a person, the 
airbag deployment is not suppressed. The system is thus 
typically biased toward airbag deployment. 

Side impact airbags are now beginning to be used on 
some vehicles. These initial airbags are quite small 
compared to the driver or passenger airbags used for 
frontal impact protection. Nevertheless, a small child 
could be injured if he is sleeping with his head against 
the airbag when it deploys and a vehicle interior 
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monitoring system is needed to prevent such a deployment in 
that event. In FIG. 9 a single ultrasonic transducer 330 
is shown mounted in the vehicle door adjacent to the airbag 
system. This device is not used to identify the object 
which is adjacent the airbag but merely to measure the 
position of the object. 

A rear-of-head detector 334 is also illustrated in FIG 
9. This detector is used to determine the distance from 
the headrest to the rear most position of the occupant's 
head and to therefore control the position of the headrest 
so that it is properly positioned behind the occupant's 
head to offer optimum support in the event of a rear 
impact. Although the headrest of most vehicles is 
adjustable , it is rare for an occupant to position it 
properly if at all. Each year there are in excess of 
400,000 whiplash injuries in vehicle impacts approximately 
90 , 000 of which are from rear impacts (source: National 
Highway Traffic Safety Administration, (NHTSA) ) . A 
properly positioned head rest could substantially reduce 
the frequency of such injuries which can be accomplished by 
the head detector of this invention. The head detector 334 
is shown connected schematically to the headrest control 
mechanism and circuitry 34 0. This mechanism is. capable of 
moving the headrest up and down and, in some cases, 
rotating it fore and aft. 

When the driver of a vehicle is using a cellular phone, 
the phone microphone frequently picks up other noise in the 
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vehicle making it difficult for the other party to hear 
what is being said. This noise can be reduced if a 
directional microphone is used and directed toward the 
mouth of the driver. This is difficult to do since 
position of drivers 1 mouths varies significantly depending 
on such things as the size and seating position of the 
driver. By using the vehicle interior identification and 
monitoring system of this invention and through appropriat 
pattern recognition techniques, the location of the 
driver's head can be determined with sufficient accuracy 
even with ultrasonics to permit a directional microphone 
having a 15 degree cone angle to be aimed at the mouth of 
the driver resulting in a clear reception of his voice. 
Such a system can also be used to permit effortless 
conversations between occupants of the front and rear 
seats. Such a system is shown in FIG. 10 which is a system 
similar to that of FIG. 2 only using three ultrasonic 
transducers 231, 232 and 233 to determine the location of 
the driver's head and control the pointing direction of 
microphone 355. Speaker 357 is shown connected 
schematically to the phone system 359 completing the 
system. 

The transducers 231 and 2 32 are placed high in the A- 
pillar and the third transducer 233 is placed in the 
headliner and displaced horizontally from transducers 231 
and 23 2. the two transducers 231 and 232 provide 
information to permit the determination of the locus of th 
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head in the vertical direction and the combination of one 
of 231 and 232 in conjunction with 233 is used to determine 
the horizontal location of the head. The three transducers 
are placed high in the vehicle passenger compartment so 
that the first returned signal is from the head. Temporal 
filtering is used to eliminate signals which are 
reflections from beyond the head and the determination of 
the head center location is then found by the approximate 
centroid of the head returned signal. That is, once the 
location of the return signal centroid is found from each 
of the three received signals from transducers 231, 232 and 
23 3 , the distance to that point is known for each of the 
transducers based on the time it takes the signal to travel 
from the head to each transducer. In this manner by using 
the three transducers plus an algorithm for finding the 
coordinates of the head center, using processor 101, and 
through the use of known relationships between the location 
of the mouth and the head center, an estimate of the mouth 
location can be easily determined within a circle having a 
diameter of about five inches (13 cm). This is 
sufficiently accurate for a directional microphone to cover 
the mouth while excluding the majority of unwanted noise. 

The headlights of oncoming vehicles frequently make it 
difficult for the driver of a vehicle to see the road and 
safely operate the vehicle. This is a significant cause of 
accidents and much discomfort. The problem is especially 
severe during bad weather where rain can cause multiple 
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reflections. Visors are now used to partially solve this 
problem but they do so by completely blocking the view 
through a large portion of the window and therefore cannot 
be used to cover the entire windshield. Similar problems 
happen when the sun is setting or rising and the driver is 
operating the vehicle in the direction of the sun. The 
vehicle interior monitoring system of this invention can 
contribute to the solution of this problem by determining 
the position of the driver's eyes. If separate sensors are 
used to sense the direction of the light from the on-coming 
vehicle or the sun and through the use of electro-chromic 
glass or a liquid crystal film, a portion of the windshield 
can be darkened to impose a filter between the eyes of the 
driver and the light source. Electro-chromic glass is a 
material where the color of the glass can be changed 
through the application of an electric current. By 
dividing the windshield into a controlled grid or matrix of 
contiguous areas and through feeding the current into the 
windshield from orthogonal directions, selective portions 
of the windshield can be darkened as desired. 

FIG. 11 illustrates how such a system operates* A 
sensor 410 located on vehicle 4 02 determines the direction 
of the light from the headlights of oncoming vehicle 404. 
Sensor 410 is comprised of a lens and a charge coupled 
device (CCD) with appropriate electronic circuitry which 
determines which elements of the CCD are being most 
brightly illuminated. An algorithm stored in processor 101 
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then calculates the direction of the light from the 
oncoming headlights based on the information from the CCD. 
Transducers 231 # 232 and 233 determine the probable 
location of the eyes of the operator 210 of vehicle 402 in 
a manner such as described above in conjunction with the 
determination of the location of the driver's mouth in the 
discussion of FIG. 10. In this case, however, the 
determination of the probable locus of the driver 1 s eyes is 
made with an accuracy of a diameter for each eye of about 3 
inches (7.5 cm) . This calculation sometimes will be in 
error and provision is made for the driver to make an 
adjustment to correct for this error as described below. 

The windshield 416 of vehicle 402 is made from electro- 
chromic glass or comprises a liquid crystal film and is 
selectively darkened at area 418 due to the application of 
a current along perpendicular directions 422 and 424 of 
windshield 416. The particular portion of the windshield 
to be darkened is determined by processor 101. Once the 
direction of the light from the oncoming vehicle is known 
and the locations of the driver 1 s eyes are known, it is a 
matter of simple trigonometry to determine which areas of 
the windshield matrix should be darkened to impose a filter 
between the headlights and the driver's eyes. This is 
accomplished by processor 101. A separate control system, 
not shown, located on the instrument panel, or at some 
other convenient location, allows the driver to select the 
amount of dark ning accomplished by the system from no 
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darkening to maximum darkening • In this manner the driver 
can select the amount of light which is filtered to suit 
his particular physiology* The sensor 410 can either be 
designed to respond to a single light source or to multiple 
light sources to be sensed and thus multiple portions of 
the vehicle windshield to be darkened. 

As mentioned above, the calculations of the location of 
the driver's eyes using acoustic systems may be in error 
and therefore provision must be made to correct for this 
error. One such system permits the driver to adjust the 
center of the darkened portion of the windshield to correct 
for such errors through a knob on the instrument panel, 
steering wheel or other convenient location. Another 
solution permits the driver to make the adjustment by 
slightly moving his head. Once a calculation as to the 
location of the driver 1 s eyes has been made, that 
calculation is not changed even though the driver moves his 
head slightly. It is assumed that the driver will only 
move his head to center the darkened portion of the 
windshield to optimally filter the light from the oncoming 
vehicle. The monitoring system will detect this initial 
head motion and make the correction automatically for 
future calculations. 

Electro-chromic glass is currently used in rear view 
mirrors to darken the entire mirror in response to the 
amount of light striking an associated sensor. This 
substantially reduces the ability of the driver to see 
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objects coming from behind his vehicle. If one rear 
approaching vehicle, for example, has failed to dim his 
lights, the mirror will be darkened to respond to the light 
from that vehicle making it difficult for the driver to see 
other vehicles which are also approaching from the rear. 
If the rear view mirror is selectively darkened on only 
those portions which cover the lights from the offending 
vehicle, the driver is able to see all of the light coming 
from the rear whether the source is bright or dim. This 
permits the driver to see all of the approaching vehicles 
not just the one with the bright lights. 

Such a system is illustrated in FIG. 12 where rear view 
mirror 460 is equipped with electro-chromic glass, or 
comprises a liquid crystal film, having the capability of 
being selectively darkened. Associated with mirror 460 is 
a light sensor 462 which determines the direction of light 
412 from the headlights of rear approaching vehicle 405. 
In the same manner as above, transducers 231, 23 2 and 233 
determine the location of the eyes of the driver 210. The 
signals from both sensor systems, 231, 232 plus 233 and 
462, are combined in processor 101, where a determination 
is made as to what portions of the mirror should be 
darkened. Appropriate currents are then sent to the mirror 
in a manner similar to the windshield system described 
above. 

Seatbelts are most effective when the upper attachment 
point to the vehicle is positioned vertically close to the 

37 



shoulder of the occupant being restrained. If the 
attachment point is too low the occupant experiences 
discomfort from the rubbing of the belt on his shoulder. 
If it is too high the occupant may experience discomfort 
due to the rubbing of the belt against his neck and the 
occupant will move forward by a greater amount during a 
crash which may result in his head striking the steering 
wheel. Women in particular experience discomfort from an 
improperly adjusted seatbelt anchorage point. For these 
reasons it is desirable to have the upper seatbelt 
attachment point located slightly above the occupant's 
shoulder. To accomplish this for various sized occupants, 
the location of the occupant's shoulder must be known which 
can be accomplished by the vehicle interior monitoring 
system described herein. Such a system is illustrated in 
FIG. 13 which is a side planer view of a seatbelt anchorage 
adjustment system. In this system a transmitter and 
receiver 520 is positioned in a convenient location such as 
the headliner located above and to the outside of the 
occupant's shoulder. A narrow elliptical beam 521 of 
energy is transmitted from transducer 520 in a manner such 
that it illuminates the occupant's shoulder and headrest. 
An appropriate pattern recognition system as described 
above is then used to determine the location and position 
of the shoulder. This information is fed to the seatbelt 
anchorage height adjustment system 528, shown 
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schematically, which moves the attachment point 52 9 to the 
optimum vertical location. 

Acoustic resonators are devices which resonate at a 
preset frequency when excited at that frequency. If such a 
device which has been tuned to 4 0 khz is subjected to 
ultrasonic radiation at 40 khz, for example, it will return 
a signal which is much stronger than the reflected 
radiation. If such a device is placed at a particular 
point in the passenger compartment of a vehicle the 
returned signal can be easily identified as a high 
magnitude narrow signal at a particular point in time which 
is proportional to the distance from the resonator to the 
receiver. Since this device can be easily identified it 
provides a particularly effective method of determining the 
distance to a particular point in the vehicle passenger 
compartment. If several such resonators are used they can 
be tuned to slightly different frequencies and therefore 
separated and identified by the circuitry. Using such 
resonators the positions of various objects in the vehicle 
can be determined. In FIG. 14 for example, three such 
resonators are placed on the vehicle seat and used to 
determine the location of the front and back of the seat 
and the top of the seat back. In this case transducers 231 
and 232, mounted in the A-pillar 662, are used in 
conjunction with resonators 641, 64 2 and 64 3 to determine 
the position of the seat. This information is then fed to 
the seat memory and adjustment system, not shown, 
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eliminating the currently used sensors which are placed 
typically beneath the seat adjacent the seat adjustment 
motors. In the conventional system the seat sensors must 
be wired into the seat adjustment system and are prone to 
being damaged. By using the vehicle interior monitoring 
system alone with inexpensive passive resonators, the 
conventional seat sensors can be eliminated resulting in a 
cost saving to the vehicle manufacturer. 

Resonators of the type described above can be used for 
making a variety of position measurements in the vehicle. 
These resonators are made to resonate at a particular 
frequency. If the number of resonators increases beyond a 
reasonable number, dual frequency resonators can be used. 
A pair of frequencies is then used to identify a particular 
location. Alternately, resonators tuned to a particular 
frequency can be used in combination with special 
transmitters, which transmit at the tuned frequency, which 
are designed to work with a particular resonator or group 
of resonators. The cost of the transducers is sufficiently 
low to permit special transducers to be used for special 
purposes. 

Another application for a resonator of the type 
described is to determine the location of the seatbelt and 
therefore determine whether it is in use. If it is known 
that the occupants are wearing seatbelts, the airbag 
deployment threshold can be increased since the airbag is 
not needed in low velocity accidents if th occupants ar 
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already restrained by seatbelts. This will reduce the 
number of deployments for cases where the airbag provides 
little or no improvement in safety over the seatbelt. FIG . 
15, for example, shows the placement of a resonator 602 
onto the front surface of the seatbelt where it can be 
sensed by the transducers 231 and 232. Such a system can 
also be used to positively identify the presence of a rear 
facing child seat in the vehicle. In this case the 
resonator 603 is placed onto the forward most portion of 
the child seat, or in some other convenient position, as 
shown in FIG. 1. 

Other uses for such resonators include placing them on 
doors and windows in order to determine whether either is 
open or closed. In FIG. 16, for example, such a resonator 
604 is placed onto the top of the window and is sensed by 
transducers 611 and 612 — In this - case -transducers 611 and 
612 also monitor the space between the edge of the window 
glass and the top of the window opening. Many vehicles now 
have systems which permit the rapid opening of the window, 
called 4 * express open 11 , by a momentary push of a button. 
For example, when a vehicle approaches a toll booth, the 
driver needs only touch the window control button and the 
window opens rapidly. Some automobile manufacturers do not 
wish to use such systems for closing the window, called 

express close 11 , because of the fear that the hand of the 
driver, or of a child leaning forward from the rear seat, 
or som other object, could get caught between the window 
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and window frame. If the space between the edge of the 
window and the window frame were monitored with an interior 
monitoring system, this problem can be solved. The 
presence of the resonator 604 on the top of the window 
glass also gives a positive indication of where the top 
surface is and reflections from below that point can be 
ignored . 

Various design variations of the window monitoring 
system are possible and the particular choice will depend 
on the requirements of the vehicle manufacturer and the 
characteristics of the vehicle. Two systems will be 
briefly described here. 

In the first example, a single transmitter/ receiver 613 
is used in place of and located centrally midway between 
the transducers 611 and 612 shown in FIG. 16. A recording 
of the output of transducer 611 is made of the open window 
without an object in the space between the window edge and 
the top of the window frame. When in operation, the 
transducer 613 receives the return signal from the space it 
is monitoring and compares that signal with the stored 
signal referenced above. This is done by processor 601. 
If the difference between the test signal and the stored 
signal indicates that there is a reflecting object in the 
monitored space, the window is prevented from closing in 
the express close mode. If the window is part way up, a 
reflection will be r ceived from the edge of the window 
glass which, in most cases, is easily identifiable from the 
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reflection of a hand for example. A simple algorithm based 
on the intensity of the reflection in most cases is 
sufficient to determine that an object rather than the 
window edge is in the monitored space. In other cases, the 
algorithm is used to identify the window edge and ignore 
that reflection and all other reflections which are lower 
(i.e. later in time) than the window edge. In all cases, 
the system will default in not permitting the express close 
if there is any doubt. The operator can still close the 
window by holding the switch in the window closing position 
and the window will then close slowly as it now does in 
vehicles without the express close feature. 

In the second system, two transducers 611 and 612 are 
used as shown in FIG. 16 and the processor 601 comprises a 
neural network. In this example the system is trained for 
all cases where the window is down and at intermediate 
locations. In operation, the transducers monitor the 
window space and feed the received signals to processor 
601. As long as the signals are similar to one of the 
signals for which the network was trained, the express 
close system is enabled. As before, the default is to 
suppress the express close. 

The use of a resonator to determine whether the vehicle 
door is properly shut is illustrated in FIG. 17. In this 
case the resonator 702 is placed in the B-pillar in such a 
manner that it is shielded by the door, or by a cover or 
other inhibiting mechanism (not shown) engaged by the door, 
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and prevented from resonating when the door is closed* If 
transducers such as 231 and 232 in FIG. 3 are used in this 
system, the closed door condition would be determined by 
the absence of a return signal from the B-pillar 702 
resonator. This system permits the substitution of an 
inexpensive resonator for a more expensive and less 
reliable electrical switch. 

The use of an acoustic resonator has been described 
above. For those cases where an infrared laser system is 
used, an optical mirror would replace the mechanical 
resonator used with the acoustic system. In the acoustic 
system, the resonator can be any of a variety of tuned 
resonating systems including an acoustic cavity or a 
vibrating mechanical element. 

A neural network, or other pattern recognition system, 
can be trained to recognize certain people as permitted 
operators of a vehicle. In this case if a non-recognized 
person attempts to operate the vehicle, the system can 
disable the vehicle and/or sound an alarm as illustrated in 
FIG. 18. In this figure the sensing transducers are shown 
as before as 231, 232 and 233, the alarm system 
schematically as 708 and the alarm as 705. Since it is 
unlikely that an unauthorized operator will resemble the 
authorized operator, the neural network system can be quite 
tolerant of differences in appearance of the operator. The 
system defaults to where a key must be used in the case 
that the system doesn't recognize the driver or the owner 
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wishes to allow another person to operate the vehicle. The 
transducers 231, 232 and 233 are sensitive to infrared 
radiation and the operator is illuminated with infrared 
waves from transducer 231. This is necessary due to the 
small size of the features which need to be recognized for 
high accuracy of recognition. An alternate system uses an 
infrared laser, which can be 231 in FIG 18, to illuminate 
the operator and a CCD device, which can be represented as 
232 in FIG. 18, to receive the reflected image. In this 
case the recognition of the operator is accomplished using 
a pattern recognition system such as described in Popesco, 
V. and Vincent, J.M. * % Location of Facial Features Using a 
Boltzmann Machine to Implement Geometric Constraints", 
Chapter 14 of Lisboa, P.J.G. and Taylor, M.J. Editors, 
Techniques and Applications of Neural Networks . Ellis 
Horwood Publishers, New York, 1993. In the present case a 
larger CCD element array containing 1000 or more elements 
would in many cases be used instead of the 16 by 16 or 2 56 
element CCD array used by Popesco and Vincent. 

Once a vehicle interior monitoring system employing a 
sophisticated pattern recognition system, such as a neural 
network or fuzzy logic system, is in place, it is possible 
to monitor the motions of the driver over time and 
determine if he is falling asleep or has otherwise become 
incapacitated. In such an event, the vehicle can be caused 
to respond in a number of different ways. One such system 
is illustrated in FIG. 19 and consists of a monitoring 
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system having transducers 231, 232 and 233 plus 
microprocessor 101 , such as shown in FIG. 7A, programmed to 
compare the motions of the driver over time and trained to 
recognize changes in behavior representative of becoming 
incapacitated. If the system determines that there is a 
reasonable probability that the driver has fallen asleep, 
for example, then it can turn on a warning light shown here 
as 805 or send a warning sound* If the driver fails to 
respond to the warning by pushing a button 806, for 
example, then the horn and lights can be operated in a 
manner to warn other vehicles and the vehicle brought to a 
stop. Naturally other responses can also be programmed. 

An even more sophisticated system of monitoring the 
behavior of the driver is to track his eye motions using 
such techniques as are described in: Freidman et al, US 
Patent 4,648,052 "Eye Tracker Communication System 11 ; 
Heyner et al, US Patent 4,720,189 "Eye Position Sensor 11 ; 
Hutchinson US Patent 4,836,670 %% Eye Movement Detector* 1 ; 
and, Hutchinson US Patent 4,950,069 "Eye Movement Detector 
With Improved Calibration and Speed 11 , all of which are 
included herein by reference. The detection of the 
impaired driver in particular can be best determined by 
these techniques* Also, in a similar manner as described 
in these patents, the motion of the driver's eyes can be 
used to control various systems in the vehicle permitting 
hands off control of the entertainment system, heating and 
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air conditioning system or all of the other systems 
described above. 

In most of the Applications described above, a single 
frequency energy was used to illuminate various occupying 
items of the passengfer compartment. This was for 
illustrative purposes only and this invention is not 
limited to single frequency illumination. In many 
applications, it is ttseful to use several discrete 
frequencies or a band of frequencies. In this manner 
considerably greater information is received from the 
reflected illumination permitting greater discrimination 
between different classes of objects. In general each 
object will have different reflectivities at each 
frequency. Also, the different resonators placed at 
different positions in the passenger compartment can now be 
tuned to different frequencies making it easier to isolate 
one resonator from another. 

Although several preferred embodiments are illustrated 
and described above, there are possible combinations using 
other geometries, sensors, materials and different 
dimensions for the components that perform the same 
functions. This invention is not limited to the above 
embodiments and should be determined by the following 
claims* 
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CLAIMS 



1 . In a motor vehicle having an interior passenger compartment, said 
compartment having contents comprising one or more objects and/or 
occupants, an interior monitoring system comprising: 
5 a) Pattern recognition means to identify at least one of- said 

contents or a part of one of said contents; 

b) Means to determine the location of said identified content or 

part of one of said contents; and 

c> Output means to effect another system within said vehicle, 
10 in response to said location determination. 



2. A system according to claim 1 in which the contents of the 
passenger compartment includes at least one occupant. 

3. A system according to claim 2 in which the pattern recognition 
means identifies a part of the or each occupant and the means to 

1 5 determine the location, determines the location of said part. 

4. A system according to claim 5 in which the part of the or each 
occupant are the eyes of the or each occupant* 

5. A system according to any one of the preceding claims in which the 
Pattern recognition means includes means to irradiate the contents of the 

20 passenger compartment. 

6. The invention in accordance with any one of claims 2 to 5 in which 
said other system is an airbag system for protection of the occupants of 
the vehicle. 

7. The invention in accordance with any one of claims 2 to 5 in which 
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said other system is a gas generator of an airbag system, said gas 
generator being of the variable inflation rate type and said invention 
further comprising means to control the inflation rate of said variable 
inflation rate inflator based on the position of said occupant. 

8. The invention in accordance with any one of claims 1 to 5 in which 
said other system is a rear view mirror. 

9. The invention in accordance with any one of claims 1 to 5 in which 
said other system is an adjustable seatbelt anchorage system. 

10. The invention in accordance with any one of claims 1 to 5 in which 
said other system is an adjustable headrest system. 

1 1 . The invention in accordance with claim 4, further comprising means 
for determining the direction of an external light source; 

said output means in response to the determined location of the 
eyes of the occupant and the direction of the external light source, 
being arranged to activate a light filter between the external light 
source and the determined location of the eyes of the occupant. 

12. The invention in accordance with claim 2, in which said pattern 
recognition means comprise means for illuminating an occupant, means 
for receiving reflected illumination from the occupant and means for 
processing the received reflected illumination to create an electronic 
signal; 

the attentiveness of the occupant being determined from 
successive determinations of the location of the occupant by said 
location determining means and said output means being arranged 
to effect the another system based on the determined attentiveness 
of the occupant. 
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13. The invention in accordance with any one of claims 1 to 12, in 
which said pattern recognition means comprise a trained neural network. 

14. The invention in accordance with claim 1, in which said pattern 
recognition means identify at least one of said contents which is an 
occupying item and said at least one occupying item has surfaces, said at 
least one occupying item being of a certain class, said pattern recognition 
means comprising receiver means for receiving radiation from said 
surfaces of said at least one occupying item and processor means coupled 
to said receiver means for processing said received radiation in order to 
create ah electronic signal characteristic of said at least one occupying 
item based on said received radiation, said signal containing a pattern 
representative and characteristic of said at least one occupying item; 

said pattern recognition means identifying the class of said at least one 
occupying item by processing said signal based on said received radiation 
from said surfaces of said at least one occupying item into a 
categorisation of said signal characteristic of said at least one occupying 
item; 

said location determining means being coupled to said processor 
means; 

said output means being coupled to said pattern recognition means 
and said location determining means for affecting the another system 
based on the categorisation of said electronic signal characteristic of the 
least one occupying item and the determined location of the at least one 
occupying item. 

1 5. The invention in accordance with claim 14, further comprising 
means for irradiating a portion of said vehicle interior passenger 
compartment in which the at least one occupying item is situated. 

16. The invention in accordance with claim 14 or claim 15, in which 



- 50 - 



said at least one occupying item is a rear facing child seat. 

17. The invention in accordance with any one of claims 14 to 16, in 
which said output means produce an output representative of the number 
of occupants in said vehicle and said at least one other system is a vehicle 
communication system. - - 

18. The invention in accordance with any one of claims 14 to 1 6, in 
which said output means produce an output representative of specific 
seat occupancy and said at least one other system is a vehicle 
entertainment system. 

19. The invention in accordance with any one of claims 14 to 16, in 
which said output means produce an output representative of seat 
occupancy and said at least one other system is a vehicle heating and air 
conditioning system. 

20. The invention in accordance with any one of claims 14 to 16, in 
which said at least one occupying item is at least an occupant, said 
invention further comprising measurement means for measuring the 
temperature of said vehicle occupant, said occupant temperature being 
output by said output means to said at least one other system. 

21. The invention in accordance with claim 15, in which said irradiating 
means are selected from the group consisting of means for producing 
acoustic radiation and means for producing electromagnetic radiation. 

22. The invention in accordance with any one of claims 14 to 21, in 
which the at least one occupying item is an occupant, further comprising 
determining means coupled to said processor means for receiving said 
electronic signal and determining the attentiveness of said occupant from 
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said electronic signal; and 

second output means coupled to said determining means and to one of 
the systems in the vehicle for affecting said one of the systems in the 
vehicle in response to the determined attentiveness of said occupant. 

23. The invention in accordance with any one of claims 1 4 to 22, in 
which said receiver means generate a signal which varies over time 
depending on the shape of the at least one occupying item. 

24. The invention in accordance with any one of claims 14 to 1 6, in 
which said at least one other system within said vehicle is selected from 
the group consisting of:- 

(i) an entertainment system; 

(ii) a distress notification system; 

Uii) a heating and air-conditioning system; 

(iv) a vehicle safety system; 

(v) a light filtering system; and 

(vj) an unauthorised user/ignition enabling system. 

25. The invention in accordance with any one of claims 1 4 to 24, in 
which said pattern recognition means comprise a trained neural network. 

26. The invention in accordance with claim 1 , in which said contents 
are of certain classes, said pattern recognition means comprising receiver 
means for receiving radiation from at least one of said contents in the 
compartment and generating a signal corresponding to the received 
radiation, said pattern recognition means recognising and thus identifying 
the class of said at least one of said contents by processing said signal 
corresponding to the received radiation into an identification of the class 
of said at least one of said contents, 

said location determining means being coupled to said pattern recognition 
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means, and said output means being coupled to said location determining 
means for affecting at least one other system within the vehicle based 
on:- 

(i) the determined location of said at least one of said contents 
within said vehicle; and 

(ii) the identification of the class of said at least one of said 
contents within said vehicle. 

27. The invention in accordance with claim 26, in which said 
recognition and identification means further comprise means for irradiating 
said at least one of said contents of said compartment. 

28. The invention in accordance with claim 26 or claim 27, in which 
said at least one of said contents comprises an occupant. 

29. The invention in accordance with any one of claims 26 to 28, in 
which said vehicle further comprises a rear seat and said at least one 
other system is selected from the group consisting of an airbag system for 
frontal impact protection of an occupant of the front seat of the vehicl 
when the occupant is seated in the front seat, an airbag system for frontal 
impact protection of an occupant of the rear seat of the vehicle when the 
occupant is seated in the rear seat and an airbag system for side impact 
protection of an occupant. 

30. The invention in accordance with any one of claims 26 to 28, 
further comprising comparison means to compare the position of the 
occupant at different times after an accident and to thereby determine 
motion of the occupant and wherein said at least one other system is a 
vehicle communication system. 

31. The invention in accordance with any one of claims 26 to 28, in 
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which said at least one other system is an airbag system and means are 
further provided to determine the velocity of the occupant from 
successive location measurements. 

32. The invention in accordance with any one of claims 26 to 28, in 
which said at least one other system is an airbag system including a gas 
generator having a variable inflation rate, further comprising means to 
control the inflation rate of said gas generator based on the determined 
location of the occupant. 

33. The invention in accordance with any one of claims 26 to 28, in 
which said at least one other system is an airbag system including an 
electronic crash sensor having a sensor threshold triggering level and 
means for setting the sensor threshold triggering level, further comprising 
means to determine seatbelt usage of the occupant, said means for 
setting the sensor threshold triggering level of the electronic crash sensor 
being coupled to said seatbelt usage determining means and being 
structured and arranged to consider the seatbelt usage to set the sensor 
threshold triggering level. 

34. The invention in accordance with any one of claims 26 to 28, in 
which the contents constitute at least one occupant, said pattern 
recognition means being structured and arranged to recognise and identify 
at least one part of the at least one occupant and said location 
determining means being structured and arranged to determine the 
location of said at least one identified part of the at least one occupant 
within said passenger compartment. 

35. The invention in accordance with claim 34, in which said at least 
one part of the at least one occupant is the head, said output means 
affecting said at least one other system based at least in part on the 
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determined location of said head of the at least one occupant. 

36. The invention in accordance with claim 34, in which said at least 
one part, of the at least one occupant are the eyes. 

37. The invention in accordance with claim 34, in which said at least 
one other system is selected from the group consisting of an adjustable 
seatbelt anchorage system, an adjustable headrest system and a window 
closing system. 

38. The invention in accordance with any one of claims 26 to 28, in 
which said signal comprises a plurality of data, all of said data being 
compared to data corresponding to patterns of received radiation stored 
within said pattern recognition means and associated with possible 
classes of contents of the vehicle. 

39. In a motor vehicle having an interior passenger compartment for 
enclosing at least one occupying item having surfaces and assigned to a 
certain class, a method for controlling a system in the vehicle based on 
the at least one occupying item comprising the steps of: 

receiving radiation from said surfaces of the at least one occupying 
item within the passenger compartment; 

processing said received radiation in order to create at least one 
electronic signal characteristic of the at least one occupying item within 
the passenger compartment based on said received radiation, each of said 
at least one signal containing a pattern representative and characteristic of 
the at least one occupying item; 

determining the location of the at least one occupying item within 
the passenger compartment based on said received radiation; 

categorising each of said at least one signal by utilising pattern 
recognition means for recognising and thus identifying the class of the at 
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least one occupying item, said pattern recognition means processing each 
of said at least one signals based on said received radiation from said 
surfaces of the at least one occupying item into a categorisation of said 
signal characteristic of the at least one occupying item; and 

affecting the system in the vehicle based on the categorisation of 
said at least one signal characteristic of the at least one occupying item 
and the determined location of the at least one occupying item. 

40. The method of claim 39, further comprising the step of: 
irradiating at least a portion of the passenger compartment in which 

the at least one occupying item is situated. 

41 . The method of claim 39 or claim 40, in which the system in the 
vehicle is a vehicle communication system, further comprising the steps 
of: 

producing an output representative of the number of occupants in 
the vehicle based on the categorisation of said at least one signal 
characteristic of the at least one occupying item; and 

controlling the vehicle communication system based on the output 
representative of the number of occupants in the vehicle. 

42. The method of claim 39 or claim 40, in which the system in the 
vehicle is a vehicle entertainment system, further comprising the steps of: 

producing an output representative of specific seat occupancy 
based on the categorisation of said at least one signal characteristic of the 
at least one occupying item; and 

controlling the vehicle entertainment system based on the output 
representative of specific seat occupancy. 

43. The method of claim 39 or claim 40, in which the system in the 
vehicle is a vehicle heating and air conditioning system, further comprising 
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the steps of: 

producing an output representative of specific seat occupancy 
based on the categorisation of said at least one signal characteristic of the 
at least one occupying item; and 
5 controlling the vehicle heating and air conditioning system based on 

the output representative of specific seat occupancy. 

44. The method of any one of claims 39 to 43, further comprising the 
steps of: 

arranging temperature measurement means for measuring the 
1 0 temperature of an occupant of the passenger compartment; and 

in addition to affecting the system in the vehicle based on the 
categorisation of said at least one signal characteristic of the at least one 
occupying item and the determined location of the at least one occupying 
item, affecting the system in the vehicle based on the measured 
1 5 temperature of the occupant. 

45. The method of any one of claims 39 to 43, further comprising the 
steps of: 

determining attentiveness of the at least one occupying item from 
said at least one signal if the at least one occupying item is an occupant; 
20 and 

in addition to affecting the system in the vehicle based on the 
categorisation of said at least one signal characteristic of the at least one 
occupying item and the determined location of the at least one occupying 
item, affecting the system in the vehicle based on the determined 
25 attentiveness of the occupant. 

f 

46. The method of any one of claims 39 to 43, in which said received 
radiation is processed to generate at least one signal which varies ov r 
tim depending on the shape of th at least one occupying item. 
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47. The method of any one of claims 39 to 46, in which said pattern 
recognition means comprise a trained neural network. 

48. The method of claim 39 or claim 40, in which the vehicle further 

- comprises a rear seat and the system is selected from a group consisting 
of an airbag system for frontal impact protection of an occupant of the 
front seat of the vehicle when the occupant is seated in the front seat, an 
airbag system for frontal impact protection of an occupant of the rear seat 
of the vehicle when the occupant is seated in the rear seat and an airbag 
system for side impact protection of an occupant. 

49. The method of claim 39 or claim 40, further comprising the step of: 
comparing the location of the at least one occupying item at 

different times to thereby determine motion of the at least one occupying 
item. 

50. The method of claim 39 or claim 40, further comprising the step of: 
determining the velocity of the at least one occupying item from 

successive location measurements. 

51 . The method of claim 39 or claim 40, in which the system is an 
airbag system including a gas generator having a variable inflation rate, 
further comprising the step of controlling the inflation rate of said gas 
generator based on the determined location of the at least one occupying 
item. 

52. The method of claim 39 or claim 40, in which the system is an 
airbag system including means for controlling the flow of gas into or out 
of an airbag further comprising the step of controlling said flow of said 
gas into or out of said airbag based on the determined location of th at 
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least one occupying item. 

53. The method of claim 39 or claim 40, in which the system is an 
airbag system including an electronic crash sensor having a sensor 
threshold triggering level and means for setting the sensor threshold 
triggering level, further comprising the step of determining seatbelt usage 
of the occupant, said means for setting the sensor threshold triggering 
level of the electronic crash sensor being coupled to said seatbelt usage 
determining means and being structured and arranged to consider the 
seatbelt usage to set the sensor threshold triggering level. 

54. The method of claim 39 or claim 40, in which the step of receiving 
radiation from said surfaces of the at least one occupying item comprises 
the step of arranging a plurality of receivers in the vehicle each at a 
discrete location, said received radiation from each of said receivers being 
processed in order to create a respective electronic signal characteristic of 
the at least one occupying item within the passenger compartment based 
on said received radiation from said receiver, each of said electronic 
signals being categorised by utilising said pattern recognition means. 

55. The method of claim 39 or claim 40, in which said pattern 
recognition means are structured and arranged to additionally use 
information from another sensor when recognising and thus identifying 
the class of the at least one occupying item. 

56. The method of claim 55, in which said another sensor is selected 
from a group consisting of a weight sensor, a seatbelt buckle sensor a 
seatbelt payout sensor, a seat position sensor, and a seatback angle 
sensor. 

57. In a motor vehicle having an interior passenger compartment having 
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(i) at least one occupant, said occupant having eyes, (ii) a windshield and 
(iii) a rear view mirror, a monitoring system comprising: 

a. Pattern recognition means to determine the direction of an 
external tight source; 

b. Means to determine the location of the eyes of said 
occupant; and 

c. Means responsive to the direction of said external light 
source and said location of said eyes to automatically 
activate a light filter between said external light source and 
the eyes of said occupant. 

58. In a motor vehicle having an interior passenger compartment 
containing at least one occupant, an occupant attentiveness detection 
system comprising: 

a. Means to illuminate said occupant; 

b. Means to receive reflected illumination from said occupant; 

c. Means to process said received illumination to create an 
electronic signal; 

d. Means to determine the attentiveness of said occupant from 
said electronic signal; and 

e. Output means in response to said attentiveness 
determination to affect at least one other system within said 
vehicle, 

59. A vehicle interior monitoring system substantially as described with 
reference to and as illustrated in any of the figures of the accompanying 
drawings. 
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